For a mobile robot to operate autonomously, it should be able to locate obstacles and steer around them as it moves within its environment. For example, a mobile robot may acquire images of its environment, process them to identify and locate obstacles, then plot a path around the obstacles identified in the images. In some cases, a mobile robot may include multiple cameras, e.g., to acquire stereoscopic image data that can be used to estimate the range to certain items within its field of view. A mobile robot may also use other sensors, such as radar or lidar, to acquire additional data about its environment. Radar is particularly useful for peering through smoke or haze, and lidar returns can sometimes be used determine the composition of objects within the environment. A mobile robot may fuse lidar, radar, and/or other data with visible image data in order to more accurately identify and locate obstacles in its environment. To date, however, sensory processing of visual, auditory, and other sensor information (e.g., LIDAR, RADAR) is conventionally based on “stovepiped,” or isolated processing, with little interaction between modules.